Reliable identification with preselection and rejection class

ABSTRACT

Disclosed is a method for classifying samples wherein the information content of a sample is divided up into two information areas. The first area contains necessary information and the second area contains sufficient information. An initially imprecise preselection concerning a plurality but extremely limited number of classes is made through the necessary information area. After said preselection, identification occurs with the sufficient information area in order to pinpoint the initially imprecise preselection in relation to an effective target class. This enables classification quality to be improved by means of new classifiers or new principal formulations of classification formulations.

A classification of data sets (e. g. picture data, speech signals) isthe basis of an “intelligent” computer performance. Numerous fields ofuse exist, e. g. industrial production, biometrical recognition ofhumans, medical picture processing, etc.

The state of the art comprises a great number of classificators, e. g.

statistical classificators (Gaussian distribution classificators)

neuronic networks

synergistic algorithms

next-neighbor classificator.

A standard literature in the field of pattern recognition is Nieman,“Klassifikation von Mustern”, Springer Verlag, 1983.

It is an object of the present invention to further improveclassification quality by providing new classificators or new basicformulations of a classification.

This object is solved by one of claim 1 and 10.

Explanations are intended to contribute to a better comprehension of thetechnical terms used in the claims.

Identification (or Classification) of n Classes:

After providing n classes from a predetermined representative off-handsample in a so-called learning process, an association/classification ofa (still) unknown pattern into a certain class is called‘identification’. By introducing a rejection threshold, a pattern may berejected as unknown. If it approximates a rejection class more closelythan one of learned and known target classes of said identification, itis classified into said rejection class. A rejection threshold and arejection class may be provided alternatively and cumulatively. Apattern is regarded as a “rejected pattern” (object or person), if all“rejections” provided (at least one of a threshold and a class) haveresponded. A precondition for a successful “identification” is that thetest pattern provides sufficient information for a clear association toone of said n classes of the learning process.

Verification:

An identification with n=1 is made by an a priori (previous) knowledgeconcerning the target class, i. e. like a binary decision, (only) anacceptance or a rejection of the test pattern (patterns used for thetest, shortly: “test pattern”) may result.

FAR, FRR, Quality Function:

FAR (false acceptance rate) designates the rate of patterns identifiedfalse; FRR (false rejection rate) designates the rate of patternsrejected false. A quality function G=G (FAR,FRR) indicates the qualityof a classification process, e. g. G=1−FAR−FRR. The more precise aclassification, the closer G approaches “one”. A weighting of FAR andFRR may have an influence if one or the other parameter FAR, FRR shallbe accentuated, e. g. by indicating an average value with weightingfactors g₁,g₂, e. g. (g₁·FAR+g₂·FRR)/(g₁+g₂). In practical applications,FRR may be of more importance, so that e. g. g₂=2 and g₁=1 may beselected to make said quality G “measurable” and to be able to compareidentifications.

The method according to the invention serves to improve classificationquality.

(a) In a first step, an ‘identification’ of n classes is provided, saididentification being improved by a double use of a provided information.For this purpose, an information content of a test pattern is split intoa necessary and a sufficient portion to be associated to a class. Withsaid necessary portion, a preselection (pre-classification) of theclasses to be considered may be effected. With said method, no precise(but rather an imprecise) classification is obtained, but the number ofclasses to be really considered for the pattern is substantially limitedor reduced. Said step provides a “better” identification (in the meaningof the above quality function G).

(b) In a second step, classification quality is improved by an(additional) rejection class. Said class serves to support a rejection,i. e. in addition to rejections obtained for instance by thresholddecisions, a particular rejection class is equally entitled with respectto the identification classes (the concrete target classes), into whicha classification may be effected. With said rejection class, an a priori(previous) knowledge concerning the objects/persons (general term:patterns) to be rejected is taken into consideration insofar as e. g. arepresentative profile of the “patterns” to be rejected is learned intosaid rejection class and is therefore known to the classificator.

The composition of the rejection class is “subject to success”, i. e.(in the meaning of the quality function) a better solution of theclassification problem has to be provided by using said rejection class.When patterns to be rejected are e. g. known, they may all be learnedinto said rejection class. But usually, only a certain portion thereofis necessary for an improved rejection class, another portion may forexample originate from a data base not relating to this problem at all.

For the selection of patterns to be rejected, selection methods may beused, e. g. testing all possibilities and using those data sets (ofpatterns) which yield the best result.

In the following, the invention(s) is (are) specified and supplementedby several embodiments.

FIG. 1 is a graphic embodiment of a classification being provided with arejection class.

A defined learning and classification process comprises the followingsteps:

Providing digital data (sets) of a classification problem.

Defining identification classes K₁, . . . , K_(n) by a representativeoff-hand sample of each class (usual learning process).

For each subset of {K₁, . . . , K_(n)} a rejection class is determined,delimiting the totality of all possible test patterns from said subset.For this purpose, at least one of the data sets of the other classes andpossible rejection candidates may be used.

Calculating a necessary information content. The features may be quitedifferent, however—in order to be suitable—they may be used for alimitation in a pre-classification 20.

By considering the necessary condition, an identification of n classesis attributed to a classification 30 of a lower degree m (at best m=1).

A second step comprises an identification/verification 40, whichadditionally is provided with a rejection class 10.

When a classification into one of said m classes K₁. . . K_(m) is made,an object or a person is regarded as recognized; when a classificationinto said rejection class 10 is made or when a rejection threshold iscrossed, said object or said person is rejected.

A pertinent example of said first step may be explained with regard tothe difference of speech recognition and speaker recognition:

Speech Recognition:

A spoken word is identified—as independently as possible from a specificspeaker—by using signal processing methods.

Speaker Recognition:

The recording of a speech signal is associated with a speaker—asindependently as possible from the content.

The object to be achieved is a reliable speaker identification servingfor example as an admission control.

In the first embodiment, the speakers have to pronounce their surname tobe identified.

A necessary precondition for identifying really e. g. Mr. Maier¹ is,that the name “Maier” has been pronounced. In a speech recognition, onlythose classes are considered which phonetically sound like “Maier”. Inthis embodiment, a reduction to n=1 is not provided, as also Mrs. Maieror Mr. Meyer, who are to be identified as well, belong to the (phonetic)sub-selection “Maier” and may be mixed up with regard to classification.Only a subsequent speaker identification results in a clear association.Therefore, the first embodiment, in relation to a speech recognition,describes a necessary information portion, to previously separate allnon-“Maier”, not phonetically sounding like “Maier”.

¹ n.b.: Maier, Meier, Mayr, Mayer, Meyer, Mayer are very common Germansurnames having several spellings, but may not be translated into anEnglish synonym. It is left in the translation as it is in the originalPCT application. (the translator).

In the second embodiment, the speakers must indicate a definite codeword, specific for the person, e. g. a sequence of numbers. With aprevious speech recognition, a reduction of the identification problemto n=1 was obtained, i. e. now, it is only necessary to effect averification based on a sufficient information content, the necessaryinformation content having been evaluated before.

Although, in prior art better solutions for speech recognitions than forspeaker recognitions are known, a mere speaker recognition quicklyreaches its limits due to restricted calculating capacity, as a learningpattern has to be compared with each class (example: bank customers).However, the FAR as well as the FRR decrease with a smaller number ofclasses. E. g. in a mere identification case of 66 persons, a system“SESAM” according to DE 44 13 788.5 (or WO 95/25316) provides a qualityfunction (G_(sesam)=1−FRR+FAR) of less than 80% in an acoustic range,whereas a verification increases the quality function to 97.2%.

A third embodiment is to describe the additional use of a rejectionclass. In a biometrical identification of persons, each authorizedperson has an identification class. A rejection class is provided by aselection of a large pool of personal data sets. Besides saididentification classes, said rejection class represents “the rest of theworld”, figuratively often described with “for the rest”. Thus,unauthorized persons which have erroneously not been rejected, areclassified into said rejection class and are thereby correctly rejected.

In case of e. g. 10 pictures (patterns) of a person to be verified and apool of 200 pictures (patterns) of a representative off-hand sample ofthe population, biometrical data of mimicry, visual aspects andacoustics are used as features. Further, an off-hand test pattern of 20pictures of the person to be verified and 100 pictures of unknownpersons are present. The pictures of “the pool” are learned into saidrejection class, leading to an optimum result on the off-hand testsample, i. e. a quality function G is maximized. It is obvious that forthe figures of this embodiment, rapid selection methods are used, averification requiring 2²⁰⁰ tests.

Until now, non-authorized persons could only be rejected with arejection threshold; however, dependent on said threshold, a high FAR orFRR and thus a low quality value G is obtained.

The object of the invention is to further improve a classificationquality of in an identification, by providing one of new classificatorsand new principle formulations of a classification statement. Theinvention proposes a classification method comprising two steps, saidmethod evaluating an information content of a pattern (object or person)in two information portions. Said first information portion comprisingnecessary information and said second information portion comprisingsufficient information. With said necessary information portion, a stillimprecise preselection concerning several, but a substantially limitednumber of classes is made. Said preselection is followed by anidentification with said sufficient information portion, to concretizesaid still imprecise preselection with regard to a real target class.

We claim:
 1. A method for classifying patterns to at least one realtarget class, said patterns having an information content and saidcontent of a pattern to be classified being actively split into firstand second information portions, said first information portioncontaining necessary information and said second information portioncontaining sufficient information, said method comprising two steps: (a)a preselection is made using said necessary information portion, saidpreselection being initially imprecise and concerning a substantiallylimited number of classes; (b) said preselection is followed by anidentification step using said sufficient information portion toconcretize said initially imprecise preselection with regard to one ofsaid real target classes contained in said limited number of classes. 2.The method of claim 1 comprising a rejection class provided as anavailable class for an initial association of such patterns which are tobe rejected with at least good probability, as a result of a previousknowledge learned into said rejection class.
 3. The method of claim 2wherein said rejection class is equally entitled in relation to theother classes of a respective method step.
 4. The method of claim 3,wherein said rejection class is provided in said second method step andis equally entitled in relation to one of said target classes.
 5. Themethod of claim 2, wherein a first number of patterns is present, beingonly partly learned into said rejection class, resulting in a secondnumber of patterns to obtain an optimized result of the second number ofpatterns, whereby a third number of patterns not being attributable tosaid rejection class is present in said first and second numbers ofpatterns, said third number being smaller than said second number andsaid second number being smaller than said first number.
 6. The methodof claim 1 wherein a previous knowledge is learned into a rejectionclass before starting said second method step, using a representativeprofile of patterns not being associatable to classes of a positivelyclassifying identification.
 7. The method of claim 6, wherein saidrepresentative profile of patterns are patterns to be reliably rejected.8. The method of claim 1 having at least two subsets within the targetclasses of the identification step, wherein an individual rejectionclass is learned or defined previously for each subset.
 9. The method ofclaim 8 wherein one of said subsets relates to said imprecisepreselection concerning the limitation of possible classes for thesubsequent identification into a respective target class as a finalclassification.
 10. The method of claim 1 wherein with said necessaryinformation, a classification problem is reduced to a still necessaryidentification problem of a considerably lower degree.
 11. The method ofclaim 10 wherein said identification is a verification of said imprecisepreselection, by evaluating only said sufficient information portion.12. Apparatus comprising a calculating unit, an input unit and an outputunit, said calculating unit being programmed to operate for classifyingpatterns to at least one real target class, said patterns having aninformation content and said information content of a pattern to beclassified being actively split into first and second informationportions, said first information portion containing a necessaryinformation and said second information portion containing a sufficientinformation, said program comprises two steps: (i) a preselection ismade using said necessary information portion, said preselection beinginitially imprecise and concerning a substantially limited number ofclasses; (ii) said preselection is followed by an identification stepusing said sufficient information portion to concretize said initiallyimprecise preselection with regard to one of said real target classescontained in said limited number of classes.
 13. The apparatus of claim12, comprising an information splitting unit, separating necessaryinformation and sufficient information to provide a first classificationunit with a first information portion and an identification unit with asufficient information portion, said two portions being smaller than thetotal information of a pattern to be classified.
 14. The apparatus ofclaim 12, wherein in addition to said at least one target class, arejection class is defined as a storage area, said rejection class beingequivalent in function with respect to said target class, but beinglarger to include all patterns not to be associated to said targetclass.
 15. The apparatus of claim 12, wherein in addition to said atleast one target class, a rejection class is defined as a storage area,said rejection class being equivalent in hierarchy with respect to saidtarget class, but being larger to include all patterns not to beassociated to said target class.